1. Field of the Invention
The present invention relates to an image processing apparatus.
2. Description of the Background Art
In response to a recent trend towards higher pixel resolution of a digital camera, higher transmission speed and higher image processing speed of captured image data have been required. FIGS. 7 and 8 each schematically show a flow of image data captured by a conventional digital camera. An interlace sensor 2 is used as a CCD sensor for image capturing in FIG. 7, whereas a progressive sensor 18 is used in FIG. 8. With reference to FIGS. 7 and 8, image processing operation of a conventional digital camera will be discussed below.
With reference to FIG. 7, when a CCD sensor for image capturing is the interlace sensor 2, the interlace sensor 2 first converts captured information as light about a subject into a digital signal which is then outputted as image data. The outputted a subject into a digital signal which is then outputted as image data. The outputted digital signal is subjected to noise control by correlated dual sampling, A/D conversion into a digital signal and the like at a signal processing unit 3, and is thereafter sent to a main processor 4. The main processor 4 comprises a sensor processing unit (hereinafter referred to as “SPU”) 6 for performing signal processing such as white balance control on the received image data. Next, the processed image data is stored in a main memory 5.
The interlace sensor 2 outputs the captured image of one frame in two fields, namely, odd and even fields. However, image processing using the SPU 6 and later units requires peripheral pixel information, and hence, information only about the odd field or about the even field is not responsive to the image processing. In response, subsequent process steps are put on hold until the interlace sensor 2 outputs the image data of two fields to reproduce the image data of one frame as original image data 16 on the main memory 5.
After the image data of one frame is reproduced as the original image data 16, a real-time processing unit (hereinafter referred to as “RPU”) 7 performs signal processing such as interpolation, color space conversion and pseudo color suppression on the image data written to a line memory 8 from the main memory 5. The image data processed by the RPU 7 is thereafter sent to an image compression and expansion unit 13 through line buffers A24 and B25 on the main memory 5. For compression into the JPEG (Joint Photographic Experts Group) format, the line buffers A and B each normally have a capacity of 8 lines.
More specifically, after the processed image data is sent to the line buffer A24, the RPU 7 continues to process subsequent image data read from the original image data 16, which processed data is then sent to the line buffer B25. The image data sent in advance to the line buffer A24 is concurrently sent to the image compression and expansion unit 13 and is subjected to processing therein. The RPU 7 continues to be operative to send subsequent image data to the empty line buffer A24, concurrently with which the image data sent in advance to the line buffer B25 is sent to the image compression and expansion unit 13. That is, the two line buffers A24 and B25 become alternately operative.
The image data read from the line buffers A24 and B25 are subjected to compression into a predetermined format such as JPEG at the image compression and expansion unit 13, and are thereafter sent as compressed image data 17 to the main memory 5. The compressed image data 17 is subjected to processing by an image display unit 19 and a signal conversion unit 20, whereby an image is displayed on a display device 21, or the compressed image data 17 is stored in a memory card through a memory card interface. The image display unit 19 uses a display buffer 29 as a working buffer memory.
Unlike the interlace sensor 2, the progressive sensor 18 as a CCD sensor for image capturing outputs the captured image of one frame at a time, and hence, generation of the original image data 16 on the main memory 5 is not required. Except for this, image processing using the progressive sensor 18 follows the same flow as the foregoing image processing using the interlace sensor 2.
As a result of higher pixel resolution, the CCD sensors 2 and 18 each have a growing number of pixels per horizontal line, reaching such a length that exceeds the horizontal line length of the line memory 8 of the RPU 7 in some cases. On the occurrence of such excess, regardless of whether the CCD sensor 2 or 18 is used, the image data outputted from the CCD sensor 2 or 18 cannot be directly processed by the RPU 7. In response, even in the use of the progressive sensor 18, the original image data 16 is generated once on the main memory 5, and thereafter the subsequent process steps are followed. More specifically, with reference to FIG. 9, the original image data 16 is divided into left and right regions to avoid excess of the pixels per horizontal line of the original image data 16 over the horizontal line length of the line memory 8 of the RPU 7. The left region including regional data P1 through P(n/2) is processed first to generate a region 27 including regional data Q1 through Q(n/2). Next, the right region including regional data P(n/2)+1 through Pn is processed to generate a region 28 including regional data Q(n/2)+1 through Qn. Finally, two regional data R1 and R2 on the left and right are synthesized to form intermediate data 26 of one frame on the main memory 5 which is then processed by the image compression and expansion unit 13.
As discussed, for compression of a subject image captured by a digital camera into a predetermined format such as JPEG to be displayed on a display device or stored in a memory card, image data transmission should be repeated between the main processor 4 and the main memory 5 through a bus 14. More specifically, a digital camera using the interlace sensor 2 requires five transmissions through the bus 14 as shown in FIG. 7. Even a digital camera using the progressive sensor 18 requires three transmissions.
As discussed, the CCD sensors 2 and 18 have a horizontal line of pixels reaching a length that exceeds the horizontal line length of the line memory 8 of the RPU 7, in which case the regional data R1 and R2 on the left and right separately processed are synthesized into the intermediate data 26 which is then held in the main memory 5. As a result, the main memory 5 should be larger in capacity than required in the case in which the CCD sensors 2 and 18 contain pixels per horizontal line which do not reach the horizontal line length of the line memory 8 of the RPU 7.
The foregoing data transmissions are handled by a DMA control unit 15 which realizes high-speed DMA transmission through the bus 14 with no intervention of a CPU. However, when the CCD sensors 2 and 18 are a 5-megapixel-class CCD sensor with 2560 pixels high by 1920 pixels wide each storing 16 bits of information, for example, the image data outputted from the CCD sensor 2 or 18 contains information as much as about 10 megabytes. In light of a recent trend towards still higher pixel resolution of the CCD sensors 2 and 18, repetition of transmission of such high-volume data causes heavily places a burden on a bus band, thus leading to increase in power consumption, lower processing speed, and the like.